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SUMMARY 


The  Air  Force  Human  Resources  Laboratory,  Operations  Training  Division  (AFHRL/OT)  is 
currently  completing  the  development  of  a  new  F-16C  simulator  with  full  f ield-of-view  visual 
display  and  no  motion  system.  This  paper  describes  the  methods  used  to  measure  the  transport 
delays  that  exist  In  this  system  and  some  of  their  effects  on  the  training  effectiveness  of  the 
simulation.  Because  these  modern  simulators  tend  to  be  complex  in  nature  and  consist  of  many 
computers  interfaced  with  each  other,  the  determination  and  measurements  of  the  transport  delays 
are  often  difficult.  The  effects  these  delays  have  on  the  simulation  of  a  high-performance, 
fighter-type  aircraft  are  also  difficult  to  determine.  The  objectives  of  this  experiment  were: 

(a)  develop  a  method  for  easily  measuring  the  transport  delays  of  an  existing  simulator  system; 

(b)  measure  the  delays  due  only  to  the  computer  hardware;  (c)  measure  the  delays  due  to  software 
and  hardware  combinations;  and  (d)  determine  the  effects  of  transport  delay,  if  any,  on  training 
in  the  simulator.  To  accomplish  these  objectives,  a  three-step  process  was  outlined  as  follows: 
(1)  design  and  construct  hardware  to  interface  with  the  cockpit  and  aero  computations  (basic 
side),  and  Interface  with  the  visual  computer  and  display  system  (visual  side)  to  provide  a 
recording  of  Inputs  and  responses;  (2)  analyze  the  delays  expected,  using  a  system  block  diagram 
and  equipment  performance  specifications,  and  compare  to  the  data  collected  In  step  1;  and  (3) 
compare  objective  flight  evaluations  of  the  simulator  using  data  collected  during  F-16C 
transition  training  conducted  at  AFHRL/OT.  Results  indicated  the  Importance  of  making  transport 
delay  measurements  on  simulation  equipment.  These  measurements  verify  that  the  device  is 
actually  performing  according  to  specifications  and,  if  not,  show  where  the  bottleneck  is 
occurring.  The  method  developed  for  measuring  these  transport  delays  is  relatively  simple  and 
Includes  a  unique  and  innovative  technique  for  determining  the  output  from  a  simulator  visual 
display. 


PREFACE 


This  paper  documents  a  presentation  given  by  Capt  Scott  J.  Horowitz  at  the  8th 
Interservice/ Industry  Training  Systems  Conference  held  in  Salt  Lake  City,  Utah,  on  18-20 
November  1986.  The  research  was  conducted  at  the  Operations  Training  Division  of  the 
Air  Force  Human  Resources  Laboratory,  Williams  Air  Force  Base,  Arizona,  under  Work  Unit 
1123-03-64,  Analysis  of  Flight  Simulator  Transport  Delay  Effects.  This  effort  Is  In 
support  of  Technical  Planning  Objective  3,  Training  Technology,  whose  general  objective 
Is  to  Identify  and  demonstrate  cost-effective  strategies  and  new  training  systems  to 
develop  and  maintain  combat  effectiveness. 
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MEASUREMENT  AND  EFFECTS  OF  TRANSPORT  DELAYS  IN 
A  STATE-OF-THE-ART  F-16C  FLIGHT  SIMULATOR 


I.  INTRODUCTION 

To  perform  research  involving  transport  delay,  it  is  first  convenient  to  define  a  few  terms 
related  to  transport  delay.  Figure  1  shows  graphically  the  definitions  of  transport  delay, 
equivalent  time  delay,  and  effective  time  delay.  These  delays  are  shown  as  a  system's  response 
to  a  step  input  and  have  the  following  physical  significance:  (a)  Transport  Delay  -  this  is  the 
type  of  delay  that  is  associated  with  pure  delay  where  the  response  is  zero  until  the  end  of  the 
delay  period,  sometimes  referred  to  as  "time  to  wiggle";  (b)  Equivalent  Time  Delay  -  this  delay 
is  determined  by  assuming  a  functional  form  of  the  system  response  (usually  a  more  simple  model 
of  the  actual  system)  and  determining  the  delay  due  to  the  term  exp(l/t);  and  (c)  Effective  Time 
Delay  -  this  delay  is  achieved  graphically  and  is  defined  as  the  intercept  on  the  time  axis  of 
the  maximum  slope  of  tne  system's  response.  For  the  simulator  experiments  conducted  at  the  Air 
Force  Human  Resources  Laboratory,  Operations  Training  Division  (AFHRL/OT),  the  type  of  delays 
investigated  were  pure  transport  delays. 
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Graphic  Definition  of  Transport  Delay,  Effective  Time  Delay, 
and  Equivalent  Time  Delay. 


The  objectives  of  this  experiment  were:  (a)  develop  a  method  for  easily  measuring  the  trans¬ 
port  delays  of  an  existing  simulator  system;  (b)  measure  the  delays  due  only  to  the  computer 
hardware;  (c)  measure  the  delays  due  to  software  and  hardware  combinations;  and  (d)  determine  the 
effects  on  training  in  the  simulator,  if  any,  of  transport  delay  (see  Figure  2).  To  accomplish 
these  objectives,  a  three-step  process  was  outlined  as  follows:  (1)  design  and  construct  hard¬ 
ware  to  Interface  with  the  cocxpit  and  aero  computations  (basic  side),  and  interface  with  the 
visual  computer  and  display  system  (visual  side)  to  provide  a  recording  of  inputs  and  responses; 
(2)  analyze  the  delays  expected,  using  a  system  block  diagram  and  equipment  performance  specifica¬ 
tions,  and  compare  to  the  data  collected  in  step  1;  and  (3)  compare  objective  flight  evaluations 
of  the  simulator  using  data  collected  during  F-16C  transition  training  conducted  at  AFHRl/OT. 
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Figure  2,  Simulator  Fidelity. 


II.  EXPERIMENTAL  SETUP 

The  interface  designed  and  constructed  for  the  basic  side  integration  was  relatively  straight¬ 
forward,  due  in  part  to  the  fact  that  the  F-16C  is  a  fly-by-wire  aircraft  and  all  information  is 
available  in  analog  or  digital  form.  An  interface  card  was  designed  and  inserted  in  the  system 
as  seen  In  Figure  3.  This  card  provided  the  pilot  input  as  a  stick  voltage  due  to  stick  pressure 
as  well  as  the  signal  the  basic  computer  sent  to  the  visual  computer  (pitch  angle  and  roll  angle). 
All  this  information  was  sent  as  analog  signals  to  the  strip  chart  recorder.  The  Interface  to 
the  visual  output  was  not  as  straightforward.  Since  there  are  delays  associated  with  the  calcu¬ 
lation  of  the  visual  scene  as  well  as  delays  due  to  the  projection  system  (cathode-ray  tube  (CRT), 
light  valves,  etc.),  It  was  decided  that  the  best  way  to  measure  the  visual  output  was  directly 
from  where  the  pilot  would  detect  the  visual  scene;  i.e.,  the  display  system  itself.  In  order  to 
accomplish  this,  a  device  was  designed  and  constructed  by  Mr.  Bill  lenenwiever  of  General 
Electric  to  convert  the  moving  image  on  a  CRT  to  an  analog  signal.  This  system  is  shown  in 
Figure  4. 
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Figure  3.  Experimental  Setup. 
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ure  4,  VIsual-to-Analog  ( V-A)  Converter 


The  visual  detector  consists  of  five  main  components:  (a)  detector,  (b)  latch  network,  (c) 
decoder,  (d)  oigital-to-analog  (0/A)  converter,  and  (e)  amplifier.  The  detector  consists  of  16 
photosensitive  transistors  mounteo  on  a  printed  circuit  (PC)  board.  Each  sensor  covers  approx¬ 
imately  12  raster  lines  on  the  CRT  monitor  (6  odd  field  and  6  even  field).  The  window  defini¬ 
tions  for  the  monitor  were  changed  in  the  software  such  that  several  scan  lines  on  the  monitor 
correspondea  to  one  scan  line  in  the  actual  viewing  field.  The  latching  network  is  connected  to 
the  visual  computer  and  is  run  by  the  60Hz  pulse  that  runs  the  system  which  synchronizes  the 
latching  network  with  the  visual  output.  The  latching  network  “holds'*  the  information  from  the 
phototransistors  for  the  whole  field  since  the  raster  only  momentarily  illuminates  the  photo¬ 
transistor  and  would  result  in  a  momentary  spiked  output  rather  than  a  continually  increasing 
output  as  the  horizon  on  the  monitor  moved.  The  information  is  then  decoded,  converted  to  an 
analog  signal,  and  finally,  amplified  for  use  by  the  strip  chart  recorder.  The  delay  in  the 
electronics  of  the  visual  sensor  account  for  approximately  12  nanoseconas  and  is  not  considered 
significant  when  compared  to  the  quantities  being  measured. 

The  software  used  in  the  F-16C  simulation  had  to  be  altered  slightly  to  provide  the  outputs 
necessary  for  determining  wnen  the  system  had  received  the  input  from  the  control  stick  ano  sent 
the  information  to  the  visual  computer.  This  presents  a  bit  of  a  problem  not  unlike  the 
Heisenburg  uncertainty  principle:  You  want  to  measure  some  quantity,  but  in  the  process  of 
measuring  it,  you  introduce  changes  and  are  now  measuring  a  changed  system.  Much  care  needs  to 
be  exercised  when  making  changes  to  the  software  of  a  flight  simulator  to  ensure  that  the  system 
is  altered  as  little  as  possible.  One  of  the  major  problems  encountered  in  the  software  modifica¬ 
tions  for  this  program  centered  around  the  fact  that  the  simulator  did  not  have  a  “perfect"  trim 
condition  when  taken  off  "freeze."  This  resulted  in  the  aircraft's  changing  attitude  without  any 
stick  input  and  thus,  made  it  difficult  to  determine  where  the  beginning  of  the  measurement  of 
delay  began.  Another  problem  encountered  was  that  in  order  to  have  a  signal  to  show  when  the 
basic  computer  finished  its  calculations  required  the  software  to  send  a  signal  to  the  D/A 
converter  which  sends  a  signal  to  the  strip  chart.  The  placement  of  this  code  in  the  simulation 
software  is  critical  and  should  be  as  close  to  the  actual  transfer  of  data  to  the  visual  computer 
as  possible.  Modifications  to  the  software  were  also  introduced  to  study  two  types  of  delay:  (a) 
delay  due  only  to  hardware  and  (b)  delay  due  to  hardware  and  software  combined.  To  measure  the 
delay  due  only  to  hardware,  the  code  is  changed  to  allow  the  stick  input  to  be  received  oy  the 
basic  computer  while  the  entire  aerodynamics  package  is  skipped.  At  the  system  interrupt,  the 
basic  computer  simply  sends  the  visual  computer  either  a  90-degree  pitch  up  or  down  signal  cor¬ 
responding  to  whether  the  stick  was  pulled  or  pushed.  This  results  in  a  step  input  at  the  stick 
providing  a  step  output  on  the  visual  system.  If  the  stick  is  driven  by  a  square  wave  generator, 
then  the  output  through  the  visual  system  will  also  be  a  square  wave  with  a  phase  shift  corre¬ 
sponding  to  the  delay  of  the  system.  The  setup  that  includes  the  software  for  the  aerodynamics 
of  the  aircraft  should  yield  the  same  transport  delays  as  long  as  everything  is  working  cor¬ 
rectly.  The  only  difference  will  be  that  a  square  wave  input  will  not  result  in  a  square  wave 
output,  as  the  aerodynamics  of  the  aircraft  will  act  as  a  filter  and  distort  the  results,  but  the 
onset  or  "time  to  wiggle"  should  remain  the  same.  If  the  test  shows  that  the  delays  were 
increased.  It  is  expected  the  increase  will  be  in  increments  equal  to  the  frame  time  of  the 
system,  as  the  software  package  may  not  have  finished  before  the  end  of  the  frame.  In  this  case 
the  "frame  drop"  will  be  detected  as  a  lack  of  output  for  one  or  more  frame  times  (33.3ms  for  a 
30Hz  system).  In  testing  the  software,  it  Is  Important  to  "exercise"  the  aero  package  by  running 
the  tests  with  the  aircraft  in  different  configurations. 

Two  methods  of  collecting  data  were  used  for  these  tests.  To  measure  hardware  delay  only, 
the  control  stick  input  was  replaced  with  a  square  wave  generator.  To  select  the  frequency  at 
which  to  drive  the  system,  one  must  first  determine  the  expected  delay  time  ano  corresponding 
frequency  of  the  system  being  measured.  In  this  case,  a  maximum  delay  of  about  150ms  was 
expected;  this  is  equivalent  to  6.66Hz.  Since  each  cycle  of  the  square  wave  will  input  both  an 
up  and  down  pitch  (right  and  left  roll),  the  signal  frequency  must  be  no  larger  than  half  the 
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system  frequency,  or  3.33Hz.  It  should  also  be  noted  that  if  the  frequency  selected  is  exactly 
an  integer  fraction  of  the  system  frequency,  the  measured  delay  will  be  a  constant  because  the 
input  will  always  be  in  sync  with  the  system.  In  order  to  measure  the  range  of  delays,  the 
signal  frequency  should  be  slightly  offset.  The  second  method  used  to  measure  delay  was  a  step 
input.  This  was  accomplished  with  the  aid  of  a  pilot  and  a  stick  cutout  switch.  The  pilot  first 
trimmed  the  aircraft  to  fly  straight  and  level.  The  next  step  was  to  turn  on  the  cutout  switch 
which  removed  the  control  inputs  from  the  system.  The  pilot  first  trimmed  the  aircraft  to  fly 
straight  and  level.  The  next  step  was  to  turn  on  the  cutout  switch  which  removed  the  control 
Inputs  from  the  system.  The  pilot  then  input  maximum  stick  deflection  (actually  pressure  on  the 
F-16C),  and  the  cutout  switch  was  deactivated.  The  result  was  that  the  system  received  a  step 
input  from  a  trimned  condition;  this  bypassed  the  problem  of  the  simulator's  inability  to  come 
off  "freeze"  in  a  trimmed  state. 


III.  DISCUSSION  OF  RESULTS 

The  first  analysis  to  be  accomplished  when  measuring  a  simulator  system  for  transport  delay 
is  to  determine  what  delays  are  expected.  In  order  to  accomplish  this,  several  items  must  be 
determined:  (a)  the  delays  of  each  of  the  system's  components,  (b)  the  type  of  visual  perception 
model  to  be  employed,  and  (c)  how  the  components  are  interfaced.  A  schematic  of  the  system  used 
in  this  experiment  is  shown  in  Figure  3,  which  includes  the  transport  delays  for  each  of  the 
system  components.  The  different  perception  models  basically  refer  to  when  to  assume  the  pilot 
has  perceived  a  change  in  the  visual  scene  (i.e.,  the  beginning  of  the  first  field,  the  end  of 
the  first  field,  or  the  end  of  the  second  field).  In  this  experiment,  the  beginning  of  the  even 
field  (the  second  field)  was  used  as  the  moment  of  perception.  Using  these  definitions  and  the 
data  in  Figure  3,  it  is  a  relatively  straightforward  task  to  add  up  all  of  the  delays  in  the 
system.  Adding  all  of  the  delays  in  Figure  3  yields  a  total  maximum  transport  delay  of  134.84ms. 
Unfortunately,  things  are  not  quite  so  simple.  In  order  to  understand  the  internal  workings  of 
this  simulator  system,  one  must  look  at  a  timing  diagram  which  shows  how  all  of  the  devices  are 
related  to  the  system  clock.  Figure  5  shows  the  timing  diagram  for  this  simulator  system.  The 
most  important  item  to  note  is  that  the  location  of  the  software  commands  for  reading  the  control 
input  is  critical  in  determining  the  expected  transport  delay.  As  shown  in  Figure  5,  this 
reading  occurs  lftns  after  the  beginning  of  the  basic  computer  interrupt  and  thus,  the  expected 
delay  should  be  10ms  less,  or  124.84ms.  Since  the  stick  input  may  occur  at  any  time,  there  is  a 
range  of  delays  that  will  be  encountered.  This  range  is  determined  by  the  length  of  the  basic 
computer  calculations,  which  for  our  system  is  33.3ms.  So,  the  total  delay  that  should  be 
expected  will  range  from  124.84ms  to  158.17ms,  or  an  average  delay  of  141.51ms. 

The  raw  data  for  this  experiment  were  collected  on  an  eight-channel  strip  chart  recorder 
running  at  200mm/sec.  The  channels  used  for  this  experiment  were  1  through  7  and  contained  the 
following  information:  (1)  10Hz  reference  clock,  (2)  pitch  input,  (3)  roll  input,  (4)  basic 
computer  pitch  output,  (5)  basic  computer  roll  output,  (6)  visual  output,  and  (7)  system  clock 
(frame  interrupt).  An  example  of  the  raw  data  collected  is  shown  in  Figure  6. 

A  sample  of  the  reduced  data  for  these  tests  is  presented  in  Figure  7.  These  figures  show 
the  transport  delay  as  a  function  of  sample  number  for  the  no  flight  equations  case.  Figure  7A 
shows  the  results  for  the  entire  system.  Figure  7B  shows  the  results  for  the  basic  computer  side 
only,  and  Figure  7C  shows  the  results  for  the  visual  side  only.  Figure  7A  shows  the  measured 
delay  varies  by  approximately  33.3ms,  as  expected,  and  shows  the  average  total  transport  delay  to 
be  148ns.  This  average  delay  is  about  6.5ms  over  the  expected  result.  Figure  7B  shows  the 
transport  delay  from  the  control  input  to  the  basic  output  is  averaging  61ms,  which  is 
approximately  5.4ms  greater  than  the  55.6ms  delay  expected.  Figure  7C  shows  the  visual  system  is 
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almost  constant  at  87ms,  which  compares  favorably  to  the  86ms  expected.  It  should  also  be 
pointed  out  here  that  the  delays  listed  in  Figure  3  for  the  devices  between  the  control  stick  and 
the  basic  computer  are  considered  worst-case  delays.  Therefore,  the  5.4ms  discrepancy  is  a 
minimum,  and  the  actual  difference  may  be  as  high  as  10ms.  This  discrepancy  was  researched  in 
some  detail,  and  the  only  conclusion  that  could  be  drawn  was  that  the  interface  between  the 
programmable  asynchronous  communication  system  (PACS)  and  the  basic  computer  is  not  operating  as 
advertised.  Figure  8  shows  the  same  results,  except  that  the  software  for  the  flight  equations 
was  included.  For  the  flight  condition  tested,  the  software  did  not  always  finish  in  time  for 
the  data  transfer  to  the  visual  system,  as  can  be  seen  by  the  spikes  in  Figure  88. 

Figure  9  shows  the  average  total  transport  delay  as  a  function  of  the  time  since  the  beginning 
of  the  experiment.  It  is  Interesting  to  note  that  the  results  show  the  system  operating  well 
outside  of  specifications  at  the  beginning  of  the  experiment  and  asymptotically  approaching  the 
design  specifications  near  the  end  of  the  experiment.  What  makes  this  result  even  more 
interesting  is  the  fact  that  the  contractors  working  on  the  simulator  maintain  that  no  changes  to 
the  system  were  made.  It  is  left  to  the  reader  to  draw  any  conclusions  he/she  wishes  from  this 
figure. 

The  final  result  that  was  obtained  concerns  the  pilots'  acceptance  of  the  simulator.  Before 
this  experiment  was  begun,  the  simulator  was  being  used  for  transition  training  and  familiariza¬ 
tion.  The  basic  response  from  the  pilots  was  that  the  simulator  did  not  fly  like  the  aircraft 
and  did  not  handle  well.  By  the  end  of  the  experiment  the  pilots  who  were  interviewed  still  said 
that  the  simulator  did  not  fly  like  the  aircraft,  but  stated  that  the  simulator  was  no  harder  to 
fly  than  the  aircraft  and  that  all  tasks  could  be  accomplished  without  much  difficulty.  Although 
this  experiment  hardly  constitutes  an  In-depth  examination  of  the  handling  qualities  of  a 
simulator  as  a  function  of  the  transport  delay,  it  does  indicate  that  a  properly  operating 
simulator  with  minimal  transport  delay  will  be  more  acceptable  to  the  pilots. 

IV.  CONCLUSIONS 

These  results  indicate  the  Importance  of  making  transport  delay  measurements  on  simulation 
equipment.  These  measurements  verify  that  the  device  is  actually  performing  according  to 
specifications  and  if  not,  show  where  the  actual  bottleneck  is  occurring.  The  method  developed 
for  measuring  these  transport  delays  Is  relatively  simple  and  includes  a  unique  and  innovative 
technique  for  determining  the  output  from  a  simulator  visual  display.  In  order  to  determine  the 
effects  of  transport  delay  on  simulator  handling  qualities,  another  experiment  is  in  progress 
that  will  utilize  an  in-flight  simulator  and  ground-based  simulators.  This  experiment  will  look 
at  how  of  varying  transport  delay  affects  flight  simulation  user  acceptance. 


< 

* 

< 

« 

i 

I 

I 

* 

1 


9 


#  of  samples 


frame  drop 


TIME  SINCE  BEGINNING  OF  EXPERIMENT  (months) 

F iyure  9.  Average  Transport  Delay  Versus  Time. 


